AutoGraph: Optimizing DNN Computation Graph for Parallel GPU Kernel Execution

Deep learning frameworks optimize the computation graphs and intra-operator computations to boost inference performance on GPUs, while inter-operator parallelism is usually ignored. In this paper, a unified framework, AutoGraph, proposed obtain highly optimized in favor of parallel executions GPU kernels. A novel dynamic programming algorithm, combined with backtracking search, adopted explore optimal graph optimization solution, fast estimation from mixed critical path cost. Accurate runtime information based Multi-Stream launched CUDA Graph utilized determine convergence optimization. Experimental results demonstrate that our method achieves up 3.47x speedup over existing methods. Moreover, AutoGraph outperforms state-of-the-art kernel launch by 1.26x.